This invention relates to image simulation for use in digital image transactions, more specifically creating and color transforming a composite image combining a computer graphics image, a natural image and written characters.
Printers, television stations, etc. do not always use images just as they are taken with a camera. They also use patch printing, chromakey synthesis and the like to produce new images by inserting one image into another. Image synthesis and color change are also conducted in image simulation for processing digital images.
This has been achieved by the following procedures:
(1) Image gathering PA1 (2) Segmentation PA1 (3) Attributes PA1 (1) Determining the direction vector for each component, because the intermingling of the components makes it difficult to separate the color into its individual components. Therefore, the pixels are plotted in RGB space and their distribution examined. PA1 (2) If the ambient color is assumed constant, then each pixel color is represented by the vector sum of the object color and the light source color, and the plotted points are distributed in the plane including the two vectors. An operator observes the pixel distribution and creates the minimum parallelogram enclosing all of the plotted points. PA1 (3) Since two sides of the parallelogram correspond to the vectors of the object color component and the light source color component, the intrinsic values can be obtained by projecting the color of each pixel on each vector and calculating the maximum value of each component. PA1 (1) to create a composite image from a CG image and a natural image; PA1 (2) to enable a CG image or a composite image created from a CG image and a natural image to be immediately used in image simulation, and wherein the color of an appointed image area can be changed; PA1 (3) to texture map an appointed image area; PA1 (4) to change the orientation of an object created by computer graphics and contained in a CG image or a composite image created from a CG image and a natural image; and PA1 (5) to use mask calculation with respect to written characters and pattern designs, without need for conducting geometrical transformation together with pasting.
The source image to be pasted and the background image are obtained. If a person wishes to create an image of a house with a car parked in front, a snapshot of the car is taken as the source image and a picture of the house is taken as the background photograph.
The region of the source image to be pasted is cut out (this region is called the "mask"). In our example this would be the region of the source image occupied by the car. Methods for creating masks are discussed in PIXEL, No. 84, pp. 181-184, under the heading of Introduction to Digital Painting for Graphic Designers.
The mask is assigned attribute information required for calculations and processing. This work is directly related to the segmentation and is conducted by a human operator defining the attribute information. A color picture is nothing more than orderly arranged color pixels. The observer of an image is able to recognize the depicted object or to imagine the object shape only because he or she has knowledge related to the shape of the object or knows how to view the object in terms of the directions of the light source and the planes. Therefore, when provided with only a natural image, the person has this knowledge.
The aforementioned prearranged processes (1)-(3) for image simulation have been manually conducted with respect to a digital image created from the natural image taken with a camera, scanner, or the like. When image assembly is conducted in image simulation, an image region is cut out as a mask, and assembly transactions, i.e. color calculations, are conducted between the cut-out region (mask) and another image (background image) in accordance with the attributes.
Images created using computer graphics (CG images) are also used to create composite images by combination with natural images. When a CG image is used in image simulation, the same prearranged processes are again required between the CG image and the natural image. This is because the purpose of computer graphics is to create CG images and the CG image normally does not become a color image until after CG color image formation.
CG image segmentation can be classified in two categories as regards complexity. The first category includes a CG image of an object, such as a car, that normally has no background, making it easy to cut out the car object region. The second category includes a CG image of a scene where a number of objects coexist, making the cutting out of a single object about as difficult as in the case of a natural image.
Color change simulation is described in Japanese Patent Hei 3-132595 corresponding to U.S. Ser. No. 07/892,962, filed Jun. 3, 1992 for "Method for Changing Color Images".
For simulation as described in Japanese Patent Application Hei 3-279263, corresponding to U.S. Ser. No. 07/964,293 the region to be changed is segmented from the region not to be changed with a mask, a rectangle is defined in the original image to be pasted, and, for enabling the rectangle to be modified and pasted in a quadrangle, four corresponding apexes are designated in the base image.
Define the coordinates (in a simultaneous coordinate system) of the four points in the image to be pasted as (xsi, ysi, 1) and the coordinates of the corresponding points in the base image as (xdi, ydi, 1), where i=1-4. Define the transformation matrix for moving (xsi, ysi, 1) to (xdi, ydi, 1) as A. This relationship is represented by Eq. 1. ##EQU1##
Thus, xsi and ysi can be used to represent (xdi, ydi, 1) as in Eq. 2. ##EQU2##
When Eq. 2 is converted to ordinary coordinates, xdi, ydi can be represented as in Eq. 3. EQU x.sub.di =a.sub.11 x.sub.si +a.sub.12 y.sub.si +b.sub.1 EQU y.sub.di =a.sub.21 x.sub.si +a.sub.22 y.sub.si +b.sub.2 (Eq. 3)
Writing Eq. 3 as a matrix gives Eq. 4. ##EQU3##
If the inverse matrix T.sub.2 of T.sub.1 in Eq. 4 can be obtained, Eq. 5 can be used to obtain a coefficient of matrix A. EQU (a.sub.11, a.sub.12, b.sub.1, a.sub.21, a.sub.22, b.sub.2, c, d)=(x.sub.d1, x.sub.d2, x.sub.d3, x.sub.d4, y.sub.d1, y.sub.d2, y.sub.d3, y.sub.d4).multidot.T.sub.2 (Eq. 5)
One method commonly used for obtaining matrices is the Gauss-Jacobi method. Obtaining the transformation matrix A by this method makes texture mapping of the appointed region possible.